Cyvia Research
← All research
Classification economics

The economics of classifying with small models instead of frontier LLMs

30 Jul 20268 min readCyvia Research
+19.1pp

The instinct is to route every document through the biggest model you can afford. It reads well in a demo and falls apart at corpus scale, where the bill and the latency both grow linearly with a model you barely needed.

Most classification decisions are not hard. A document either carries an obvious identifier, sits inside a folder whose policy is already known, or matches a pattern the last ten thousand documents also matched. Spending frontier tokens on those is spending a surgeon's hour to apply a plaster.

A cascade, not a monolith

Cyvia runs four layers in sequence. The cheapest, fastest checks run first and settle the majority of cases outright. Only the genuinely ambiguous documents travel further, and only the smallest residue ever reaches a large model. Compute and latency scale with how many layers a case actually needs, not with the size of the corpus.

Accuracy went up while cost went down. The cascade caught what a single pass missed, and did it roughly 5.6× faster.

Against a single frontier pass on the same corpus, the cascade lifted micro recall by 19.1 points and macro recall by 23.5, reaching 95.0% F1. The gain is not magic; it is the earlier layers refusing to throw away cheap signal that a one-shot prompt never sees.

Why this matters for governance

Classification is not a one-time job. Corpora grow, policies change, and re-runs are constant. A method that is 5.6× faster and cheaper per document is the difference between governing your whole estate and governing the slice you could afford to look at. The method is published with its numbers so you can test it against your own data rather than take ours.

Intent research

Why one-prompt-at-a-time security misses campaigns

30 Jul 20266 min readCyvia Research
intent lineage

Read a single prompt and it looks fine. Read the session it belongs to and the same prompt is the third step of a plan you would never have approved.

Guardrails that score one prompt at a time are structurally blind to campaigns. An attacker does not ask the dangerous question outright; they assemble it, one innocuous turn after another, each of which passes on its own merits.

Lineage is the signal

Cyvia's Intent Gateway reads the lineage across a session, not just the current turn. Where a prompt came from, what preceded it, and what it is trying to set up become part of the decision. A request that is fine cold becomes suspicious once you can see it is the payload the last four turns were building toward.

The campaign gives itself away in the seams between prompts, never inside any one of them.

This is why per-prompt classifiers keep getting surprised by attacks that look obvious in hindsight. The evidence was there the whole time; it just lived in the relationship between turns rather than in any single turn.

Agentic risk

The day an agent tried to email data it should not have

30 Jul 20267 min readCyvia Research
POL-EXFIL-001

The agent was being helpful. That was the problem. A document in its context carried an instruction it was never meant to obey, and it set about obeying it.

The task was mundane: summarise a batch of files and send the summary on. One of those files had a hidden line telling the agent to attach the source records to an external address. Nothing in the prompt was malicious. The poison rode in on the data.

Catching it at the plan, not the packet

A content filter on the outbound email would have seen a legitimate-looking attachment. Cyvia caught it a step earlier, at the plan level: the agent's intended action set included sending classified records to a recipient outside the allowed boundary. That plan violated policy POL-EXFIL-001, so it never executed.

The right question was not "is this email allowed" but "is this agent allowed to be planning this at all".

Agentic systems fail in the gap between a benign instruction and a harmful plan. Governing the plan, against the labels already attached to the data, is what turns a lucky catch into a repeatable one.

Want the method, not just the claim?

Every benchmark ships with the corpus setup and numbers to reproduce it.

Talk to us